Skip to content

y-script-lint and y-help#71

Merged
solsson merged 28 commits intomainfrom
y-script-lint
Mar 20, 2026
Merged

y-script-lint and y-help#71
solsson merged 28 commits intomainfrom
y-script-lint

Conversation

@solsson
Copy link
Collaborator

@solsson solsson commented Mar 20, 2026

y-script-lint and y-help

Static analysis tooling for y-* scripts, with CI gating that prevents degradation without requiring all ~210 scripts to pass upfront.

Why

We have ~210 y-* scripts across repositories with inconsistent conventions, no dependency tracking, and no way to prevent regressions. We can't e2e test utility scripts because they must support macOS + Linux, amd64 + arm64. Lint gives us a safety net that works on all platforms.

What

New scripts:

  • y-script-lint — static analysis for y-* scripts. Checks shebang, header, help handler, shellcheck, npx/eval usage. Parses the YHELP variable from source to extract summaries and dependencies without executing scripts. Detects y-* invocations for dependency tracking. Recognizes trivial binary wrappers and skips help requirements for them.
  • y-script-lint-compare — compares two lint index files. Detects degradation (a check that passed before now fails) and new script failures. Deleted scripts are ignored.
  • y-help — reads the lint index and prints one line per script: name, help summary, NOLINT marker if applicable. Supports substring filtering.

New docs:

  • Y_SCRIPT_AUTHORING.md — conventions for writing compliant scripts. Covers the YHELP variable pattern, help subcommand, OS compatibility (macOS 14.8+, Debian Trixie+, Ubuntu 24.04+), and what each lint check expects.

CI:

  • lint.yaml workflow runs on PRs and push to main. Uses --fail=degrade with cached baselines so existing failures don't block, but no branch can make things worse.
  • images.yaml now requires lint to pass before building.

How it works

y-script-lint never executes scripts. All checks are source parsing: grep, awk, shellcheck, wc. It writes a JSON index to ~/.cache/ystack/y-script-lint.json with per-script metadata (language, summary, checks, declared and detected dependencies, repo root).

Note that for js scripts the scope is to complement for example eslint and only check y-script conventions.

--fail=degrade manages two cached baselines (main and branch). On each run it compares the new index against the best available baseline. A check that was true before must still be true. New scripts only need to pass static checks (FAIL level), not help conventions (WARN level). Cache only saves on success so failed runs never corrupt the baseline.

Current ystack results: 34 pass, 3 fail (pre-existing eval usage), 37 warn (missing help).

Also included

  • --dependencies-add flag writes detected y-* invocations into a script's YHELP Dependencies section

solsson and others added 27 commits March 20, 2026 07:33
Two-phase convention checker for y-* scripts:
- Phase 1: language detection, header/debug checks, help handler detection,
  shellcheck integration, npx/eval checks. Never executes scripts.
- Phase 2: sandboxed --help execution (env -i, temp HOME, timeout 5s,
  unshare --net on Linux with fallback). Parses summary and Dependencies.
- --check exits 2 on static failures (CI gate), WARN for help issues
- --json writes .y-script-lint.json for future y-script-index
- TODO: macOS sandbox, eslint integration for Node.js scripts

First run on ystack: 73 scripts, 4 static failures (pre-existing eval/shellcheck),
68 help warnings (expected backfill work).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Detect #!/usr/bin/env node as "node" language
- Detect #!/usr/bin/env -S node --experimental-strip-types as "typescript"
- Skip header/debug checks for non-shell scripts (null in JSON)
- Node.js help handler: process.argv.includes('--help'/'-h')
- Language-aware eval check: eval( for JS/TS, bare eval for shell
- Language-aware npx check: // comments for JS/TS, # comments for shell
- Skip .spec.js/.spec.ts/.test.js/.test.ts files

Tested on all three repos:
- ystack: 73 scripts (all shell), 9 pass, 4 static failures
- checkit: 137 scripts (16 node), 36 pass, 18 static failures
- bots: 3 scripts (1 typescript), 2 pass, 1 static failure

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Without arguments, scans all PATH directories containing y-* scripts.
Groups output by directory with headers. Deduplicates by script name
(first PATH entry wins, matching shell resolution). Supports multiple
explicit DIR arguments. Writes per-directory .y-script-lint.json.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Default 5 seconds. Passed directly to timeout(1).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Output format:
  [y-script-lint] /path/to/bin (N scripts)
    y-example  bash 42L  OK
    y-broken  bash 15L  FAIL
      uses eval
      shellcheck --severity=error failed
  Total: N  Passed: N  Failed: N  Warnings: N  Skipped: N
  Wrote /path/to/bin/.y-script-lint.json

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Each issue line is now prefixed with its severity:
  y-bin-download  bash 189L
    WARN missing DEBUG pattern
    FAIL uses eval

Script name line no longer carries a status label. Clean scripts
have no children in the tree.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Trap SIGINT to exit 130 immediately (Ctrl+C was blocked by set -e
  and subshell pipe chains)
- Wrap y-shellcheck in timeout using same Y_SCRIPT_LINT_TIMEOUT_S
  (default 5s per file) to prevent hangs on phase 1

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previous trap waited for hung children to finish before exiting.
New trap resets itself then sends SIGINT to the process group (-$$),
killing all children (shellcheck, timeout, grep) immediately.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
timeout sends SIGTERM which can be ignored by child process trees
(e.g. y-vault calling y-bin-dependency-download making network requests).
Add --kill-after=2 to both sandbox and shellcheck timeouts so they
SIGKILL after 2s grace period if SIGTERM is insufficient.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rewrite Y_SCRIPT_AUTHORING.md:
- Lead with quick reference showing compliant bash/node/typescript templates
- Encourage help as subcommand (first arg) over --help flag
- Introduce YHELP variable convention for static help text extraction
- Document y-script-lint checks table with FAIL vs WARN levels
- Remove execution-dependent indexing, describe static-only strategy
- Drop -h from y-script-lint's own interface, use help subcommand

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
macOS 14.8+, Debian Trixie+, Ubuntu 24.04+.
Documents BSD vs GNU userland incompatibilities (sed -i, stat, etc.)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- date -Iseconds unavailable on macOS, document portable alternative
- Dependencies: section left empty in examples (maintained by tooling)
- Remove bold markdown styling throughout

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…-add

No script execution at all. All checks are source parsing:
- YHELP variable parsed from bash (single-quoted) and node (template literal)
- Summary extracted from first non-empty line of YHELP
- Dependencies: section parsed from YHELP
- y-* invocations detected by grepping source (comments excluded)

--dependencies-add writes detected y-* deps into the YHELP Dependencies:
section of scripts that already have a compliant section. Uses line-number
insertion to preserve surrounding whitespace. Idempotent.

JSON output adds detected_dependencies alongside declared dependencies.
tool_version bumped to 2.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Index (.y-script-lint.json) is now always written when jq is available,
not gated on --json flag. The --json option is removed.

Fix: last line used && chain which caused exit 1 under set -e when
--check was not set. Replaced with if/then.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TODO_Y_SCRIPTS_LINT.md rewritten to reflect static-only implementation.
Removed: phase 2 sandbox, unshare, env -i, macOS sandbox TODO,
timeout design, help execution, prereq-before-help workarounds,
dependency tree diagram, open questions about sandbox.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Detect the y-bin-download wrapper pattern: <=8 lines, y-bin-download
call, "$@" passthrough, no conditionals/loops/functions. These scripts
delegate to the wrapped binary's own help.

21 wrappers detected in ystack, all 8 lines. y-turbo (26L, has extra
logic) correctly excluded. Passed: 10 -> 31.

Also: prefix index write path with [y-script-lint].

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
y-help reads .y-script-lint.json from all PATH bin/ directories and
prints one line per script: name, help summary, NOLINT if applicable.
Supports optional substring filter argument.

y-script-lint now writes help_line and lint_ok to the index:
- YHELP scripts: description extracted from "y-name - description"
- Binary wrappers: "toolname binary wrapper"
- Others: null (y-help shows "no help section")

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
y-script-lint y-turbo        # lookup by name in PATH
y-script-lint ./bin/y-turbo  # explicit path
y-script-lint /abs/path/y-x  # absolute path

Single-script mode: no index written, exit 0 if lint passes,
exit 1 for warnings, exit 2 for static failures.

Also detect help) case pattern (the YHELP subcommand convention)
in addition to existing --help patterns.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shows ~/path/to/Y_SCRIPT_AUTHORING.md when any script has
FAIL or WARN, in both single-file and batch modes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All scripts from all directories merged into one index file.
Each script entry includes parent (repo root) as a property.
Removes per-directory .y-script-lint.json files from bin/.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
parent: the directory containing the script (bin/)
reporoot: git repo root if the directory is in a git repo, null otherwise

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
--fail=all: overwrites index, exits 2 if any static failures (same as --check)
--fail=degrade: writes new index alongside old, calls y-script-lint-compare

y-script-lint-compare takes two index files and exits 2 if:
- Any check that passed (true) in old index now fails (false) in new
- Any new script (not in old index) has a failing check
Checks that were already false or null are ignored.

On success in degrade mode, new index promotes to replace old.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Manages two baselines in ~/.cache/:
  ystack-script-lint.main.json    - saved on main/master builds
  ystack-script-lint.branch.json  - saved on branch builds (on success)

Baseline selection: branch > main > none (skip compare).
Y_SCRIPT_LINT_BRANCH env overrides git branch detection.

CI needs only: cache the two baseline files + run --fail=degrade.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace declare -A (associative array, bash 4+) with string-based
dedup using case pattern matching.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
GNU timeout is not available on macOS by default. The missing command
caused run_shellcheck to always exit non-zero, reporting all shell
scripts as shellcheck failures. Run shellcheck without timeout when
the command is not found.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Follow XDG convention of using a subdirectory under ~/.cache/.
Rename files from ystack-script-lint.* to script-lint.* since the
ystack prefix is now in the directory name. Update workflow cache
paths accordingly. First CI run will skip degradation check (no
baseline at new path) then resume normally.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Timeout was introduced for the original execution-based design where
scripts were run to capture help output. Shellcheck is static analysis
and does not need a timeout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@solsson solsson merged commit cd25d52 into main Mar 20, 2026
1 check passed
@solsson solsson mentioned this pull request Mar 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant